Refresh ROADMAP to drop phase numbering and reflect shipped state#313
Refresh ROADMAP to drop phase numbering and reflect shipped state#313
Conversation
Rewrites ROADMAP.md to organize work as Current State / Recently Shipped / Shipping Next / Under Consideration / AI-Agent Track / Long-term. Drops top-level phase numbering. Absorbs dCDH into the shipped estimator list. Promotes future-estimator candidates from the 2025-26 methodology research (including the inverse-SDiD "no untreated group" estimator, efficient staggered DiD, distributional DiD for staggered, LP-DiD, few-treated-units inference, Riesz-representation sensitivity, compositional-change inference, and the triple-difference covariate audit). Promotes the AI-agent track to a named long-term arc. Updates companion docs (docs/business-strategy.md Section 8, README.md dCDH section, docs/practitioner_decision_tree.rst, docs/choosing_estimator.rst, docs/api/chaisemartin_dhaultfoeuille.rst, docs/api/efficient_did.rst, docs/survey-roadmap.md, diff_diff/guides/llms-full.txt) to remove stale phase-deferral language now that the deferred items have shipped. Corrects EfficientDiD covariate-path documentation (the doubly-robust path is shipped; several docs still said "not yet supported"). Fixes the broken TODO.md anchor link into survey-roadmap.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Overall Assessment Executive Summary
Methodology
Code Quality Performance Maintainability
Tech Debt Security Documentation/Tests
Path to Approval
|
…acebo SE contract; rescope Under-Consideration entry
EfficientDiD efficiency claim: scope the unqualified "achieves the
semiparametric efficiency bound" language to the no-covariate path in
README.md, docs/api/efficient_did.rst, and diff_diff/guides/llms-full.txt.
The doubly-robust covariate path uses a linear OLS outcome regression; per
docs/methodology/REGISTRY.md L899-L903 and the class docstring in
diff_diff/efficient_did.py L139-L146, this preserves DR consistency but
does not generically attain the efficiency bound unless the conditional
mean is linear in the covariates.
dCDH placebo SE contract: correct the inconsistent placebo-NaN statements
in README.md and docs/choosing_estimator.rst; tighten diff_diff/guides/
llms-full.txt. The actual contract (per docs/methodology/REGISTRY.md L546
and diff_diff/chaisemartin_dhaultfoeuille.py): single-period DID_M^pl
(L_max=None) has NaN SE because the per-period aggregation path has no
influence-function derivation, with NaN-consistent inference even under
n_bootstrap > 0; multi-horizon dynamic placebos DID^{pl}_l (L_max >= 1)
have valid analytical SE via the placebo IF and bootstrap SE when
n_bootstrap > 0.
ROADMAP.md rescope: the "Efficient staggered DiD" Under-Consideration
entry incorrectly framed shipped staggered EfficientDiD as future work.
Rewrite as "Nonparametric / flexible outcome regression for EfficientDiD
DR covariate path" - the actual gap vs the Chen-Sant'Anna-Xie 2025 Section
4 proposal, where nonparametric OR (sieve / kernel / ML) would close the
efficiency gap the current linear OLS OR leaves open on the covariate
path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
/ai-review |
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment Executive Summary
Methodology
Code Quality
Performance
Maintainability
Tech Debt
Security
Documentation/Tests
Path to Approval
|
…ing surfaces; update dCDH class docstring Remaining EfficientDiD surfaces the prior pass missed: - ROADMAP.md line 18 (Current State estimator list) previously described EfficientDiD as "semiparametrically efficient with doubly robust covariates" - unqualified, and contradicted ROADMAP.md line 104 which correctly describes the covariate-path efficiency gap. Rewritten to reserve the unqualified bound claim for the no-covariate path and describe the DR covariate path with the linear-OR caveat. - docs/choosing_estimator.rst lines 363-374 previously said "maximum statistical efficiency" while advertising covariate support. Scoped the efficiency claim to the no-covariate path and expanded the covariate-path note with the DR / linear-OR caveat from REGISTRY.md L899-L903. dCDH class docstring (autodoc source): - diff_diff/chaisemartin_dhaultfoeuille.py lines 294-326 still opened with "de Chaisemartin-D'Haultfoeuille (dCDH) estimator - Phase 1." and listed "Phase 1 deliverables" / "Phase 1 contract" / "reserved for a future phase." Because docs/api/chaisemartin_dhaultfoeuille.rst renders `.. autoclass::` on this class, the built API page was still showing stale phase-deferral language. Rewrote the docstring to describe the shipped contract (headline DID_M plus multi-horizon DID_l, covariate adjustment, trends, non-binary treatment, HonestDiD integration, and survey support - only `aggregate` on fit() still raises NotImplementedError). Did not touch the other Phase-N references in the file (internal code comments and runtime-visible error messages), which are out of scope per the refresh plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
/ai-review |
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment ✅ Looks good Executive Summary
Methodology
Code Quality No findings. Performance No findings. Maintainability No findings. Tech Debt No findings. I did not see new deferred-work items that need Security No findings. Documentation/Tests
|
…orrect stale EfficientDiD cluster comment - docs/api/efficient_did.rst comparison table: the "Efficiency" row cell still read "Achieves semiparametric bound" unqualified, even though the surrounding page note, registry, and class docstring scope that claim to the no-covariate path. Updated to match those sources and mirrored the DR caveat in the Covariates row. - diff_diff/guides/llms-full.txt EfficientDiD constructor comment: the `cluster: str | None = None` line was still annotated "Not yet implemented," contrary to diff_diff/efficient_did.py L154-L160 which documents shipped cluster-robust SEs (Liang-Zeger on EIF values) and cluster-level multiplier bootstrap. Updated the comment to describe the actual shipped behavior so the runtime LLM guide no longer misleads agent-facing consumers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
/ai-review |
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment ✅ Looks good Executive Summary
Methodology No findings. I cross-checked the changed EfficientDiD and dCDH claims against the Methodology Registry and live class docstrings, including the EfficientDiD DR-efficiency caveat and the dCDH single-period versus dynamic placebo SE contract. Code Quality No findings. The touched Python change in diff_diff/chaisemartin_dhaultfoeuille.py:L294-L337 is docstring-only. Performance No findings. No runtime-affecting code paths changed. Maintainability No findings. Tech Debt No findings. I did not see new deferred work that needed separate tracking in Security No findings. Documentation/Tests No findings. For a docs refresh plus one docstring-only Python touch, the lack of test changes is appropriate, and the previously flagged EfficientDiD doc drift is now addressed. |
…oe-ligature The UTF-8 roundtrip test in tests/test_guides.py used '\u2014' (em-dash) as the non-ASCII fingerprint in llms-full.txt. The roadmap-refresh pass replaced em-dashes with hyphens throughout the guide (per the project's hyphens-over-em-dashes style), which removed the fingerprint and broke the test. The guide still contains '\u0153' (oe-ligature, from the author name "D'Haultfoeuille" in the dCDH section), which is the remaining non-ASCII character and a more stable fingerprint since it's tied to a paper citation rather than stylistic punctuation. Updated the test accordingly. No behavior change; this is a test-side assertion update to match the guide's current character set. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
/ai-review |
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment ✅ Looks good Executive Summary
Methodology
Code Quality
Performance
Maintainability
Tech Debt
Security
Documentation/Tests
|
Absorbs #312 (sdid scale fix), #313 (roadmap refresh + dCDH docstring rewrite), and #314 (within_transform convergence warnings) from main. Conflicts resolved in: - diff_diff/chaisemartin_dhaultfoeuille.py: took main's comprehensive Phase 1-3 feature list in the class docstring but merged in the PR #311 group-vs-PSU bootstrap-clustering framing and the replicate-weight survey-support line. Kept the PR #311 'user-specified cluster= not supported + automatic PSU-level under survey_design' wording for the cluster= parameter docstring (strictly more accurate than main's 'always clusters at the group level' text). - diff_diff/guides/llms-full.txt: kept main's more detailed placebo SE contract paragraph (which already distinguishes single-period NaN from multi-horizon analytical/bootstrap) and appended the sup-t / shared-weights / cross-horizon coverage details from the PR #311 update. Kept the PR #311 survey_design signature comment that mentions TSL + replicate + PSU bootstrap. Full regression across touched areas: 336 + 324 passing.
… remove deprecated SyntheticDiD params Package four merged PRs (#312 SDID catastrophic cancellation at extreme Y scale, #313 roadmap refresh, #314 FE imputation non-convergence signaling, #315 Frank-Wolfe SC weight solver non-convergence signaling) as 3.1.2. Also remove the SyntheticDiD(lambda_reg=...) and SyntheticDiD(zeta=...) kwargs, which have been deprecated with DeprecationWarning since v2.3.1 (2026-02-10) in favor of zeta_omega / zeta_lambda; their warning messages announced removal in v3.1. Passing the old kwargs now raises TypeError at __init__ and ValueError: Unknown parameter at set_params. Internal ridge-regression helpers that accept a lambda_reg parameter (compute_synthetic_weights, rank_control_units, Rust FFI bindings) are unaffected. Version strings bumped in diff_diff/__init__.py, pyproject.toml, rust/Cargo.toml, and diff_diff/guides/llms-full.txt. CHANGELOG populated with Fixed / Changed / Removed sections and comparison-link footer. TODO.md's "Deprecated Code" entry removed now that the task is done. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per CI review feedback (#316): removing public kwargs under a patch version violates Semantic Versioning, which CHANGELOG.md explicitly claims to adhere to. Restore lambda_reg and zeta handling in SyntheticDiD.__init__ and set_params as warning-only, and bump the removal target in the DeprecationWarning text from "v3.1" to "v4.0.0". The 3.1.2 release now carries only the four fix/doc PRs (#312 SDID scale, #313 roadmap, #314 FE imputation convergence, #315 Frank-Wolfe convergence) with no breaking changes. - diff_diff/synthetic_did.py: restore deprecated kwargs + warnings (v4.0.0 text) - tests/test_methodology_sdid.py: restore TestDeprecatedParams class + set_params deprecation test - tests/test_estimators.py: restore test_deprecated_params - CHANGELOG.md: drop Removed section; add Changed entry documenting the v3.1 -> v4.0.0 bump in the removal target - TODO.md: restore Deprecated Code section with v4.0.0 removal target and SemVer rationale Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
docs/survey-roadmap.md#deferred-work-consolidatedin TODO.md to point at the existing#current-limitationssection.Methodology references (required if estimator / math changes)
Validation
Phase Nstrings across the repo are all in whitelisted build-milestone history docs (docs/methodology/REGISTRY.md, docs/methodology/continuous-did.md, docs/performance-plan.md, docs/survey-roadmap.md historical sections) or internal TODO.md tech-debt entries that are out of scope.diff_diff/efficient_did.py(covariates parameter onfit()is shipped with doubly-robust sieve-based path).Security / privacy
Generated with Claude Code